Decison Boundary and hyperbolic unfolding¶

Decision boundary of a neural network¶

Dataset with two classes

Train a neural network on the binary classification task

In [7]:
# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data_circ, label_circ, n_epochs=15)
epoch train_loss valid_loss accuracy time
0 0.814834 0.792440 0.516000 00:01
1 0.796542 0.750172 0.516000 00:00
2 0.573485 0.481312 0.977000 00:00
3 0.398306 0.355876 1.000000 00:00
4 0.338907 0.327376 1.000000 00:00
5 0.322854 0.319389 1.000000 00:00
6 0.317590 0.316259 1.000000 00:00
7 0.315475 0.314866 1.000000 00:00
8 0.314508 0.314196 1.000000 00:00
9 0.314028 0.313856 1.000000 00:00
10 0.313778 0.313679 1.000000 00:00
11 0.313646 0.313585 1.000000 00:00
12 0.313577 0.313540 1.000000 00:00
13 0.313546 0.313523 1.000000 00:00
14 0.313538 0.313520 1.000000 00:00

Contour plot of the neural network function

The decision boundary is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary¶

First Approach: Sampling algorithm¶

  • Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
  • Implementation by Matteo.

Gradient flow method¶

  • We are pushing points to the decision boundary.
  • We are using automatic differentiation of neural networks.
  • The algorithm is fully implemented in Pytorch; hence it can be enormously accelerated via GPUs.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding¶

(joint work with Matthias Kemper)

  • We transform the euclidian metric conformally with factor $1/(nn-\frac 12)$.
  • The resulting metric space is (Gromov-)hyperbolic with the decision boundary as the boundary at infinity.
  • Geodesic rays converge to the boundary at infinity.
  • We use numerical methods for solving geodesic equations (second-order differential equations).

Computed decision boundary with dataset¶

Entanglement problem for tori¶

Decision boundary for entangled tori¶

Decision boundary for unentangled tori¶